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DE.SCRIPTION 



Method and Sys-beza of Weighted Context Feedback for Result 
Improvement in Information Retrieval 



BACKGROUND OF THE INVENTION 

The present invention relates to the field of document based 
information retrieval in the intranet and the Internet domain;, 
and in particular to a method for re-ranking an existing result 
list of documents. 

Nowadays digital information systems provide easy access to 
large amounts of information. For example, users can access 
great quantities of information in databases on a network or 
even in personal computers. Mere access to large amounts of 
information has only limited value, however, without knowing 
what is useful about the inf oirmation. 

Searching for a certain information in a huge amount of almost 
unstructured information can be a very cumbersome task. As well 
as the probability is relatively high that the desired 
information is somewhere existent in an existing collection of 
information and potentially can be found, it is at the same time 
covered up with all that additional unwanted information. 

To retrieve a desired piece of information there have been 
proposed several methods.. A first approach is to structure raw 
data before a search is started. Structuring the raw data in 
advance can be done by introducing a set of taxonomies (like 
"cars", "health", "computers" etc.) and assigning each retrieved 
docxoment into one or more of thosfe categories what can be 
performed in advance, e.g. by an administrator. 
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It is noteworthy that the term ^'document" is understood in the 
present context in the broadest sense i.e. including any 
continuous amount of information. 

Before a search is started, the user has to preselect one or 
more categories and thereby reduces the possible amount of 
returned information, because only the information stored in the 
selected categories is returned- A simple way to accomplish this 
is for example having , several different indexes for the search 
engine. The user can then select either one or more indexes. as a 
basis for the search. 

The drawback of the above described approach is that extra 
effort is necessary for preparing the raw data by defining the 
taxonomy set and by assigning each document into one or more 
categories (or indexes). Since the information most often is of 
a dynamic nature, an update of new information or categories (or 
both) has to-be done on a regular basis. Further there is a 
certain chance that some information is lost because of a wrong 
assignment to categories or a missing selection by the user. 

A second approach is to structure a search result after- having 
finished the search. Structuring the result of a search is 
either based on the input the user made when he started the 
search with -some query terms, or an attempt is made to 
dynamically find similiarities inside of the documents and group 
(cluster) ^them together. 

The second approach can.be implemented by way of clustering the 
results which means finding "categories" dynamically by looking 
. for similiarities inside of the returned documents. This can be 
achieved according to several different criteria, for example by 
scanning for lexical affinities (anyhow related expressions) and 
bundling those documents that have a certain similiarity. 
Thereby the total list of returned documents is split into 
several non-overlapping clusters that contain documents which 
are assumed to deal with the same or similiar context. 
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The drawback of the above second approach is that the search 
engine has no idea which context the user is really looking .for. 
Moreover, the clustering of the documents is performed on the 
basis "of word tuples that occur with a certain relation to each 
other in the documents. Ambiguous terms can cause some ' documents 
to be scattered all over the clusters, although from the user 
point of view they deal with the * same context. To find those 
documents the user has to open and read lots of uninteresting 
information. 

An alternative way to implement the second approach, i.e. 
structurung the search result after having finished the search, 
is to sort the found docimients in a descending order, derived by 
some comparable criterion. This method is commonly known as 
"Ranking" . The appearance of the search terms in each of the 
documents is a measurement for the importance of the individual 
document. All values are normalized so the higher the rank value 
is, the more importance ,is assumed. 

Various different algorithms are used to determine the 
individual rank value, most often the document sizes, the index 
sizes, the total number of returned information or other 
criteria are considered. 

Still another way to implement the second approach is refining 
the search by adding more precise search terms known as "Narrow 
Query". The user starts the search process with an initial query 
and examines some of the returned documents. For each document 
he assigns a relevance value that reflects if the appropriate 
document is of high or low value. The search engine scans the 
marked documents for terms that occur with a high frequency and 
uses those terms to synthesize a new query. This new query 
favors the high frequent terms of the documents with a high 
relevance and excludes the terms of the documents with ■ a low 
relevance. 
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But its drawback* is that for good results the user has to 
examine and mark a lot of documents, otherwise the refined query 
is more or less random. 

The predescribed approaches have common the drawback that only 
the entered search terms can be taken into account. Since search 
terms often are ambiguous, they can occur in various totally 
different context and cause a lot -of unwanted information to be 
returned- If, in addition, there are only a few terms entered, 
there is a high probability that lots of documents get the same 
high rank value. 

Further, in an article by R, Fagin ^nd E. L. Wimmers entitled 
"Incorporating user preferences in multimedia queries", and 
published in Proc. 1997 International Conference on Database 
Theory, pp. 24 7-2 61,. according to a third approach, it is 
proposed to weight the search terms in information retrieval 
with the help. of user feedback and to allow- to apply weights to 
any sort of rules. In particular, a formula is described to 
re-sort and re-rank the result list of a query according to the 
•weighting of the original search terms by the user. 

The drawback of the above 'third approach is that from common 
experiences in' information retrieval it is known that the 
typical query consists of very few search terms ( < 2 terms) . In 
the wast majority of all searches trying to apply any weighting 
to them would have very little impact. The main reason for' the 
- fact that the average ^search ia made with only one or two terms 
is the lacking of conception of the user what additional context 
information he had to enter to improve the query and filter the 
unwanted documents. Following this scenario the user would have 
to read some of the returned documents first to obtain some 
context information which he in turn could then use to create 
additional query terms. Only with a minimum set of -query terms 
that reflect the desired context this method would make sense. 
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Finally, all of the above cited prior art approaches have cornition 
the drawback that, if the preparation of the raw data as* 
described above is not feasible or in a situation, where 
information can be lost because of a user error, e.g. by- 
selecting a wrong index, only a method that does a post search 
improvement is acceptable . 

Thereupon, all of the described approaches require a relative 
high effort with opening and reading the contents of documents, 
thereby wasting time with useless information. 



SUMMARY OF THE INVENTION 

The object of the present invention therefore is to provide an 
information retrieval method and system which presents only ^ the' 
desired information without wasting a user's time with opening 
and reading documents of no value. 

Another object is to provide a method and a system for ranking 
information comprised of a list of documents which presents the 
desired information on top of a ranking hierarchy. 

Still another object is to minimize the possibility of user 
errors, in particular with respect to generating search queries. 

These problems are solved by the features of the independent 
claims. Advantageous embodiments of the invention are subject 
matter of the dependent claims. 

The invention accomplishes the foregoing by gathering context 
information from the documents, generating at least one rank 
criterion from the context information and re-ranking the 
documents, based on the at least one rank criterion. 



The concept underlying the invention is a post search 
improvement, particularly a human feedback to improve search 
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results. It is emphasized hereby that the proposed re-ranking is 
only based on the results of a first search procedure. 

The criterion for re-ranking hereby consists of terms themselves 
found in the documents. These terms contain information which 
allow to feedback to the context of the underlying documents. 
These terms, in the following designated as ^'context terms ^\ are 
used. in a weighting formula in order to separate documents with 
a matching context from docioments with a dismatching context. 

4 

The ambiguity of search terms which causes unwanted information 
to be returned, in a preferred embodiment, is solved by means of 
context information which is used to filter documents with 
unwanted context. In another embodiment a context weighting can 
be performed on a client data processing system, whereby no 
iterative and time consiiming access to a search engine is 
necessary. 

Using lexical affinities to be evaluated as a criterion for 
re-ranking', is only exemplarily and can be accomplished also by 
other means that are able to provide a notion of context. A 
suitable approach could be to use ^features' that have been 
extracted from the documents or to simply create statistics of 
words' . ^Features' are names of people, places, companies etc. 
and can be presented as a context indication very similar to 
lexical affinities. Especially when several extracted features 
are grouped and weighted together, the ,context definition would 
be very precise. "^Word statistics' describe^ the frequency a 
given word is occuring with in a given document. 

In case one finds several different words in a dociament at least 
a minimum amount of times - no matter if they' occur with a 
certain proximity to each other like with lexical affinities - 
and the same is true for another document, it is assxomed that 
both documents are' very close to each other concerning their 
context. 
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Both infoirmation (features as well as word statistics) can also 
be created by the search engine and transferred to the client 
together with the result list. 



Advantageously, the number of unnecessarily opened documents is 
hereby essentially reduced since an appropriate weighting of the 
context terms allows to precise the right context for the 
desired information. 



BRIEF DESCRIPTION OF THE DRAWINGS 

The invention will be understood more readily from the following 
detailed description when taken in conjunction with the 
accompanying drawings, in which 

Fig. 1 is a combined block and flow diagram depicting a 

Search process with, weighted context feedback by ^ 
Lexical affinities, in accordance 'with the invention; 

Fig. 2 is a screenshot of a possible implementation of this 
method; and 

Fig, 3 ' is a flow diagram depicting an embodiment of the 
re-ranking mechanism proposed by the invention. 



DETAILED DESCRIPTION OF THE DE^WINGS 

Referring to Fig. 1, a combined block and flow diagram is 
depicted which shows a typical search with weighted context 
feedback by lexical affinities. .The scenario described in the 
following assumes that it. is accepted to start a search with 
rather unprecise search terms and to try to structure the huge 
amount of returned data according to context information. 
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A user 100 starts a" search by entering search term(s) • The 
search term(s) is (are) transferred to- a search engine 110 which 
generates a result list 120 ranked by the search tenn(s) • The 
search engine 110, in parallel, automatically , retrieves context 
information 130 from returned result list 120 which is related 
140 to the original list of documents. The context information, 
in a preferred embodiment, • is based on lexical affinities which 
will be discussed in more detail later. 

The search engine 110 presents 150 the context information 130 
to the user 100 and asks for a feedback. The user 100 performs a 
weighting 160 of the presented context information 130 in a 
range from ^ important ^ to ^non-important ^ . 

The result list 120 is then re-ranked 170 with the user-weighted 
context information 18 0 to increase the "rank distance" of 
important and non important documents. The re-ranking is 
performed by utilizing a context ranking formula which is 
described in more detail below. The documents that are now on 
top of the list (highest context-weighted ranking value) 
represent the desired information. 

Optionally, the re-ranking can be performed iteratively whereby 
the modified list of documents ranked by the user-weighted 
context information 180 is created new 190 with different 
weightings, based on the original list, and re-ranked again 
performing the predescribed steps. 

The underlying re-ranking algorithm is based on an adaptation of 
the prementioned Fagln's and" Wimmers formula. The way to 
generate context information is the ' extraction of "lexical 
affinities". This method produces pairs of terms that are found 
in a certain relation within the documents. If one, for example, 
is searching for the term "display", that term could occur 
together with the terms "window" or "show" or "computer" or 
various other combinations. Each . combination- is an indication 
for the context the documents are dealing with. 



In the formula of R. Fagin and E. L. Wimmers, the- following 
definitions are assumed in the following. 



Let xl,...,xm be the terras' that represent the context 
information. The function f(xl,...,xm) is then the ranking 
function that returns a rank value for a given document with 
respect to these tearms- xl, . . . ^ xm. 9= (61^ . . • , Om) is the overall 
weighting for all of the m terms and 0i is the individual weight 
for term xi. Under the assumption that 91^... ^0m are all non- ' 
negative, 9l>=...>=9m and they all sum to one^ where we use the 
weighted ranking formula according to Fagin and Wimmers which 
is : 

(91 - 92)f(xl) + 2(92 - 93)f(xl,x2) 

+3(93 - 94) f (xl,x2,x3) + . . .+m 9m f (.xl, . . . , xm) (1.) 

A scenario where all m context terms get a different rating 
would increase the necessary calculating effort to the 
theoretical maximum of m. According to .equation (1) we had. to 

calculate the ranking value for each document m times, i.e. 

# , •* 

f (xl) , f (xl, x2) , f (xl,x2,x3) , . . . , f (xl/ • • . /xm) , 

since the expressions (9n - 9n+l) in equation (1) would all 
become non-equal to zero. Besides the enormous visualization 
effort to allow a user to choose m different ratings for- all the 
m context terms, it seems _very unlikely , that anybody would 
really do this. An approach where "for example three different 
rating levels are selectable is much more practical. 

Let the rating levels be ^high^, ^medium ^ and ^low,^ 
and the relation between those three rating levels 
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Ga = 2 eb and Qc = 0.5 eb 



During further evaluation with more samples of test documents it 
might turn out^ that a stronger spreading of those three levels 
is more useful;, such as 9a=46b and 6c == 0.25 9b or even more. 

Now the rating selections of all- of the context terms made by 
the user can be divided into 

a terms rated with ^high^ 

b terms rated with ^medium ^ 

c terms rated with ^low^ (3) 

where all selections sum -up to the total number of context terms 

a+b+,c=m (4) 

Because of the assumption that all 91 0m are non- negative 
and sum to one, it can be written 

a 9a + b 9b + c 9c = 1 and 

9b = (1 - a 9a - c Gc) / b. 

Because of (2) it follows 

9b = (1 - 2a Bb - 0.5c 9b) / b 

\ 

and finally 



9b. = 1 / (2a + b + 0.5c) 



(5) 



Allowing only three discrete ranking levels, one gets only three 
groups of Gn sorted according to their importance to the user. 
The first ^a^ context terms are rated high, the next ^b^ context 
terms are rated medium and the last ^c^ context terms are rated 
low. 

01 ==...,= 9a and 
Ga+l- = . . . = 9a+b and 

Ga+b+l = = Sm (6) 

Using equation (6), equation (1) can be simplified into 

a{9a - 9a+l)- f(xlyx2, ...,xa) 
+ (a+b) ((9a+b - Ga+b+l) f (xl, x2, . .-. , xa+b) 
+ (a+b+c) 9c f (xl,x2, . . • ,xa+b+c) (-7). 

In case of the only three relevance values ^a\ ^b'*, and ^c^, 
9a+l = 0b, 0a +b = 0b and 0a+b-fl = 0c/ one can substitute 

a (0a - 0b) f(xl,x2, •..^xa) 
+■ (a+b> (0b - 0c) f (xl,x2, . . .,xa+b) 
+ (-a+b+c) 0c f (xl, x2, . . . ,xa+b+c) (8). 

This has to be converted into a normalized form with only 
dependencies to the number of context terms rated with the three 
discrete rating levels and the rank functions of the appropriate 
context terms. 

Following equation (2) one can substitute equation (8) with 



a 0b f (xl, . . . , xa) 
+ 0,5 (a+b) 0b f (xl, . . , ,xa+b) 
+ 0.5 (a+b+c) 0b f (xl, . . . , xa+b+c) 



(9) 
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and according to equation (5) one finally obtains' 



[2a f (xl, . . 
+ (a.+b) f(xl. 



/ • • • / 



. ,xa) 



xa+b) 

. , xa+b+c) ] / (4a+2b+c) 



+ (a+b+c) f(xl 



f • • 



(10) 



Equation (10) calculates the relevance of a document with 
respect to the context terms xl,...,xm when a, b and c are the 
nximber of terms .that have been assigned a high (a) / medivim. (b) 
and low (c) relevance and f (xl, . . . , xa) ^ f (xl xa+b) and 
f (xl, xa+b+c) are the partial relevance functions of this 
document with respect to a subset of context terms. 

In comparison to equation (1) , one now has to call the ranking 
function only three times with different parameter lists which 
saves a considerable amount of processing effort. The additional 
factors in equation (10) are necessary to fulfill the three 
desiderata of equation (1) . For the further details concerning 
these factors it is referred to the prementioned article by 
Fagin and Wimmer. 

A first way i,s to feed back the weighting information to the 
Text Search Engine and calculate the new rank value for all - 
documents of the result list. This reveales some major problems 
as it is currently not possible to call a ranking function with 
new parameters (the context terms) on the basis of a prior 
search, because the statistics of these* new parameters are not 
available. Pnly the original search terms, that are known during 
the search process, are complemented by the statistics 
information of the search indexes and could then be used for a 
ranking. 

Besides this problem, it would also require some additional 
communication with the search engine that increases the response 
time to the user. 
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Therefore it is proposed a new and very efficient ranking 
algorithm to be used solely on the client side without any. 
further access to the Text Search Engine, The processing effort * 
ds relatively low^. so that it can also be used on socalled "thin 
clients" and a quick response to the user is guaranteed. 

The new ranking algorithm is based on the "absolute" ranking of 
the prior search that was derived from the query terms only. 
Depending on the occurrence of the context terms in a given 
document and their importance for the user a new rank value is 
calculated: 

Let Rd be the ^^absolute" rank' value of a given document ^d^ that 
resulted from the search, Td = (xl,,..,xn) is the tupel of 
context terms that are contained in document ^d\ then is 

f d (xl, . . • , xn) = Rd if xl,...,xn element of Td 



and 



f d (xl, , , . , xn) • = 0 if xl,...,xn non-element of Td (11) 

This new ranking equation (11) is used together with the 
adaptation of Fagin's and Wimmer' s weighted ranking (10) and 
results in a new rank value for each document. The desired 
effect is a change in the result list order. Those documents 
that contain the preferred context terms will appear on top of 
the list since their new rank value is higher than before. The 
documents that contain only the context terms that got a low 
relevance from the user will be pushed to the end of the list 
since their rank value is lowered. 



The above described (re-) ranking algorithm can be implemented in 
a Java environment with the help of Visual Builders. For the 
special purpose of weighted context feedback a new Java Bean was 
developed and seemlessly integrated into the Java Beans 
component model architecture. 
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Fig. 2 depicts a screenshot of a possible implementation of this 
method. The right side of this screenshot shows the result of a 
search in a table format. According to the ranking of the result 
list by the query term (in this case it was the single word 
"display") i. the documents are sorted top-down. 

On the left side there is the visualization of the Context 
Feedback. The left column with the header title "Importance" 
holds three graphical buttons on each line, the right column 
with the header title "Context" holds the pairs of lexical 
affinities on each line. The table is scrollable up and downward 
to display all the context information and the corresponding 
weighting. 

Immediately after having performed a search, the right column is 
filled with all context information that could be extracted from 
all documents of the, result list, the weighting of the 
individual context information in the left column is all set to 
medixom. 

By clicking .the plus or minus symbols .the weighting for each 
pair of lexical affinities can be modified. Leaving some entries 
in the "medium" state is also possible. As soon as the user is 
done with the weighting, he presses the "Rank new" button and 
the information is sent to the search client to do a weighted 
re-rank. Thiis causes the right result list table to refresh its 
contents with the new ranking order. 

Fig. 3 shows the mechanism that leads to a new ranking value 
^Rd_new' for each document ^d' contained in a result list. As 
already mentioned, the number of iterations per individual 
document, depends on the number of .discrete weighting levels that 
are offered to the user. Following the example depicted in Fig. 
2 with three different ratings ( ^high' , ^ medium' and ^low' ) one 
has to calculate up to three portions per document that make up 
the new rank value. 



For each document it is started with its ^absolute' rank value 
of the unweighted result list ^Rd' and set 300 the initial value 
of ^Rd_new' equal to ^0' . Further it is checked 310 whether any 
of the lexical affinities that have been weighted ^high' are 
contained" in document . If true 320, the new ranking value 
^Rd_new' is set 330 to ^2*a*Rd' ( ^a' is the number of lexical 
affinities weighted with ^high' ) , otherwise ^Rd_new' remains 
unchanged ' 340 . 

« 

In a next iteration 350 one performs a similar calculation for 
all the lexical affinities that have been rated ^high' and 
^medium' , and finally a third iteration 360 takes into account 
all the ^high' , ^medixim' and ^low' rated lexical affinities. The 
last step is a normalization of the new rank value ^Rd_new' • 

It is noted hereby that the above described mechanism for 
re-ranking documents can be applied .to any text documents or 
non-text documents, i^e. all kinds of * documents where a ranking 
principally can be performed. The term ^Mocuments""^' is understood- 
in its broadest sense and means an arbitrary allied amount of 
information. 



\ 



-CLAIMS Qi J^j^. 

1. A method for ranking a list of documents, in 
particular for re-ranking an existing result list of 
documents, comprising the steps of 

/ ■ ■ 

/ 

gathering context information from the 
docioments; 

generating at least one rank criterion 
from the cofitext information; 

ranking the documents, based on the at 
ieast one rank criterion, 

2. Method according to claim 1, whereby utilizing lexical 
affinities as the context information, 

3. . Method according to claim 1 or 2, whereby utilizing 

features as the context information. 

4. Method according to any of claims'! to 3, whereby 
utilizing word frequency statistics as the context 
information-. 

5. Method according to claim 2, whereby extracting 
lexical affinities from the documents in order to 
obtain the context information. 

6. Method according to- claim 3, whereby extracting 
features from the documents in order to obtain the 
context information 

7. Method according to claim 4, whereby extracting word 
frequency statistics from the documents in order to 
obtain the context information 
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Method according to any of claims 1 to 7, comprising 
the further step of weighting of the context 
information by a weighting function • 



9. Method according to claim 8, whereby providing 
discrete ranking levels, 

10. Method according to any of the preceding claims for 
re-ranking an existing list of text documents, 
comprising the steps of 

detecting of lexical affinity terms 
contained in the documents; 

presenting the lexical affinity terms to 
a user; 

gathering user preferences for the lexical 
affinity terms; 

re-ranking the documents based on the user 
preferences. 

11. Method according to any of the preceding claims for 
re-ranking an existing list of text documents, 
comprising the steps of 

detecting of feature terms contained in the 
documents; 

presenting the feature terms to a user; 

gathering user, preferences for the feature terms; 

re-ranking the documents 'based on the user 
preferences. 



Method according to. any of the preceding claims for 
re-ranking an existing list of text documents, 
comprising the steps of 

creating of word frequency statistics from the 
documents; 

presenting the words with a minimum frequency- to 
a user; 

« 

.gathering user preferences for the presented words of 
a minimiom frequency; 

re-ranking the documents ' based on the user 
preferences . 

Method according- to any of the preceding claims, 
whereby the re- ranking is based on the original 
ranking position .of the documents. 

Method according to any of the preceding claims, 
whereby using ranking equation (eq. (11) ) together 
with a weighted ranking equation (eq. (10)). 

System for ranking a list of documents, in particular 
for re-ranking an existing result list of documents. 
Comprising means . for gathering context information 
from the documents, for generating at least one 
rank criterion from the context information and for 
re-ranking the documents, based on the at least one 
rank criterion. 

System according to claim 15, comprising means for 
extracting lexical affinities from the documents in 
order to obtain the context information. 

System according to claim 15 or 16, comprising means 



for weighting of the context information by- a 
weighting function. 
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Disclosed is a method and a system for re-ranking an existing 
result list of dociiments. A user (100) starts a search by 
entering search term(s) . The search term(s) is (are) transferred 
to a search engine (110) which generate^s a result list (120) 
ranked by the search term(s) • The search engine (110) in 
parallel, automatically retrieves context information (130) from 
returned result list (120) which is related (140) to the 
original list of documents. The search engine (110) presents ' 
(150) the context information (130) to the user (100) and asks 
for a feedback. The user (100) performs a weighting (160) of the 
presented context information (130) in a range from ^important 
to ^non-important^. The result list ' (120) is then re-ranked 
(17 0) with the user-weighted context information (180) to 
increase the "rank distance" of important and non important 
documents. The documents that are now on top of the list 
(highest context-weighted ranking value) represent the desired 
'information. The underlying re-r^nking algorithm is based on an 
adaptation of a formula by Fagin and Wimmers . The way to 
generate context information is the extraction of "lexical 
affinities". This method produces pairs of terms that are found 
in a certain relation within the documents - 



(Fig. 1) 
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FIG. 2 
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